MEASURING AND DETECTING DIFFERENTIAL ITEM FUNCTIONING IN CRITERION-REFERENCED LICENSING TEST A Theoretic Comparison of Methods

The validity of a measurement instrument depends on the quality of the items included in the instrument. The overall aim was to compare methods for detecting and measuring differential item functioning, DIF, in order to find a suitable method for examining DIF in a dichotomously scored criterion-referenced licensing test. The methods were discussed with respect to whether they are parametric, the nature of the matching score, if they can handle dichotomously and polytomously scored items, if they can test and/or measure DIF, and if they can detect both uniform and non-uniform DIF. The methods were also discussed with respect to whether they could handle the cut-off score in particular and the sample size requirements. The results show that there is not one method that can be recommended because many of them rely on strong assumptions which need to be examined and fulfilled before they can be recommended. It was recommended that an empirical study comparing the Mantel-Haenszel, logistic regression, log linear models and an IRT method is performed. Finally, the concluding remarks provide a discussion of guidelines for what to do if an item displays DIF in a test.